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Abstract — In this paper, we study the capacity of the two-user 
multiple-input multiple-output (MIMO) linear deterministic in- 
terference channel (IC), with possible correlations within/between 
the channel matrices. The capacity region is characterized in 
terms of the rank of the channel matrices. It is shown that linear 
precoding with Han-Kobayashi type of rate-splitting, i.e., splitting 
the information-bearing symbols of each user into common and 
private parts, is sufficient to achieve all the rate pairs in the 
derived capacity region. The capacity result is applied to obtain 
an achievable rate region for the double-unicast networks with 
random network coding at the intermediate nodes, which can be 
modeled by the two-user MIMO linear deterministic IC studied. 
It is shown that the newly proposed achievable region is strictly 
larger than the existing regions in the literature. 

I. Introduction 

The two-user interference channel (IC) models the com- 
munication between two source-receiver pairs via a common 
channel. As there is no cooperation between any of the 
sources and receivers, the transmission from one source to 
its corresponding receiver is viewed as interference by the 
other source-receiver pair. The capacity for the general two- 
user IC is a long-term fundamental open problem since first 
studied by Shannon in [1]. The best achievable rate region 
to date is established by Han and Kobayashi p), where a 
common-private rate splitting technique is employed to enable 
the receiver to partially decode and subtract the interference. 
It was later pointed out in [3] that the Han-Kobayashi region 
matches the capacity region for a class of deterministic ICs. 
Recent breakthroughs in studying the capacity of two-user IC 
shows that a simple linear deterministic model captures the 
key properties of the Gaussian channel |4j and it leads to 
capacity characterization within a constant number of bits (5). 
Moreover, the linear deterministic channel model is closely 
related with the degrees-of-freedom (DoF) characterization of 
the two-user Gaussian IC as both of them focus on the high 
signal-to-noise ratio regime where noise is de-emphasized in 
order to get a better understanding of the interference (6). 
In fact, as pointed out in J5), the capacity region of the 
deterministic channel is, when properly scaled, equal to the 
generalized DoF region. 
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The DoF characterization of the two-user MIMO Gaussian 
IC is given in [7| with the assumption that the channels 
are nondegenerated, i.e. all channel matrices are full rank 
and independent of each other. Moreover, it is shown that 
zero forcing (ZF), which is normally a suboptimal strategy, is 
sufficient to achieve all DoF. Unfortunately, the result obtained 
in J7J is no longer applicable if the channel matrices are 
correlated and/or rank deficient. In this paper, we generalize 
the results in J7) by removing the assumptions on the channel 
matrices. Specifically, we study the capacity of the two-user 
MIMO deterministic IC, i.e., the IC studied in J7) but with 
the additive noise term set to zero. In contrast to J7J where 
all channels are assumed to be of full rank and independent, 
we consider the more general case that the channel matrices 
may be rank deficient and/or correlated with each other. This 
channel model is of theoretical interest by itself and also 
renders its application in obtaining an achievable rate region 
for the double-unicast networks as discussed later. The exact 
capacity region (or equivalently the DoF region if the noise 
term is non-zero) is characterized in terms of the rank of the 
channel matrices. The capacity achieving scheme is given by 
linear precoding together with Han-Kobayashi type of rate- 
splitting, i.e., the data symbols are split into a common part, 
which is decodable at both receivers, and a private part, which 
is decodable at the intended receiver only. Furthermore, the 
precoder consists of a random spreading matrix, which maps 
the data symbols into a subspace of higher dimension, and a 
ZF precoding matrix, which effectively pre-cancel the private 
symbols. Note that the capacity achieving scheme is quite 
simple and efficient due to its linear precoding and decoding 
processes. 

As an application for the capacity results derived for the 
two-user deterministic IC, we obtain a strictly enlarged achiev- 
able rate region as compared to existing schemes for the 
double-unicast networks. Both the two-user ICs and double- 
unicast networks share the similarity that each source is 
intended to send an independent message to its corresponding 
destination and it causes interference to the other source- 
destination pair. However, different from wireless ICs where 
the channels are determined by nature (e.g., channel gain, 
fading and so on), in double-unicast networks, the signals 
from the source pass through a set of intermediate nodes 
before arriving at the destination. As a result, the processing 
strategies used by the intermediate nodes directly affect the 
achievable rates of the network. It was recently shown that 
by employing network coding at the intermediate nodes, the 
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throughput can be significantly improved as compared to 
traditional techniques such as routing [8|. For single-session 
networks, it has been shown that random network coding is 
capacity achieving (9). However, for double-unicast networks, 
the optimal network coding strategy remains unknown. In this 
paper, with all the intermediate nodes performing random lin- 
ear network coding, we show that the input- output relationship 
of the double-unicast networks can be modeled by the two- 
user linear deterministic IC. As such, the capacity results we 
derived can be directly applied to obtain an achievable region. 

There are some existing works on the achievable rate 
region characterizations for the double-unicast networks (8), 
(TO), (TT). In fTO) , the problem was formulated as a linear 
programming problem by packing butterfly structures in the 
network. However, this approach is limited since only XOR 
operation is allowed in the butterfly structure. In (8), an 
achievable region was obtained by using the so-called "rate- 
exchange" method, where starting from the single-user rate 
for one of the users, a non-zero rate for the other user is 
achieved by directly sacrificing the single-user rate via some 
interference nulling schemes. In [11], another rate region was 
obtained by using some precoding techniques. As stated in 
[TT|, the region obtained in J5) does not contain that given in 
[TT|, and vice versa. In this paper, we will show analytically 
that our proposed region by utilizing the linear deterministic 
IC model contains both that in (8) and (TT). 

The rest of this paper is organized as follows. Section |n| 
introduces the system model. The main results are presented 



in Section III which characterizes the capacity region of the 
two-user linear deterministic IC. In Section [IV| the obtained 
capacity results are utilized to derive an achievable region for 
the double-unicast networks and the comparison with existing 
works is given. Finally, this paper is concluded in Section [V| 
Notations: Throughout this paper, ]R nxm denotes the space 
ofnxm real matrices and F™ xm represents the space ofnxm 
matrices in a finite field of order q. Vectors are represented 
by boldface lower-case letters, e.g. v. Matrices are denoted by 
italicized capital letters, e.g. A. A T denotes the transpose of A. 
Onxm represents a zero matrix of size n x m and the subscripts 
are omitted when there is no ambiguity from the context. The 
range (or column space) and null space of a matrix A are 
denoted by 11(A) and Af(A), respectively. 
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Fig. 1. Two-user MIMO linear deterministic IC 



We are interested in determining the number of independent 
symbols that can be simultaneously and reliably transmitted 
from si to ti and s 2 to t 2 , denoted as Ri and R 2 , respectively. 
The point-to-point capacity of the channel from Si to U is 
determined by the rank of the channel matrix, i.e., Ri = 
rank(if^), i e {1,2}. Note that when all the channel matrices 
are randomly generated (thus full rank) and independent of 
each other, the region of R\,R 2 is equivalent to the DoF 
region of the two-user Gaussian IC and is characterized by 

Q 



Ri < min{rai, n\] 
R 2 < min{ra 2 ,n 2 } 
Ri + R2 < min{rai + m 2 ,n 1 + n 2 , 

max(rai, n 2 ), max(m 2 , ni)} 



(2) 



However, for general ifn, Hi 2 , H 2 i and H 22 that may be 
correlated and/or rank deficient, the region given by |2]) is no 
longer applicable. Therefore, a more general result than ([2]) is 
necessary. 

Before presenting the main results, the following assump- 
tions are made without loss of generality: 



II. System Model 

Consider the two-user linear deterministic IC as shown in 
Fig[T] with sources denoted by s\ , s 2 and destinations by t\ , t 2 , 
respectively. The input-output relationships are given b)Q 



yi = #11X1 + #i 2 x 2 
y 2 = #21X1 + # 22 x 2 



(1) 



where Xi G M 
vectors by si 



x 2 e 



represent the independent input 



and s 2 , respectively; Hij £ 



)7ii Xrrij 



{1,2} denotes the channel matrix from to tj\ and G 
is the channel output at the i th 
recovering x^ only. 



receiver who is interested in 



The result can be extended to complex- valued channels as well 



rank[i7n H\ 2 ] = n\ 
rank[77 2i H 22 ] = n 2 



rank 
rank 



H 2 \ 

H12 
H 22 



rrii 



rn 2 



(3) 
(4) 

(5) 
(6) 



The above assumptions can be validated by showing that the 
capacity region will be unaffected by removing the dependent 
received or transmitted symbols. For example, if the matrix 
[if 11 if 12] does not have full row rank, certain symbols 
received by t\ are linear combinations of the rest and therefore 
they can be discarded without losing any information. 
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III. Main Results 

Theorem 1: For the two-user MIMO linear deterministic IC 
given in ([I), the capacity region is characterized by 



R x <rank(i7n) 

R 2 <rank(7J 2 2) 
R\ + R2 <n\ +7112 — rank(i^i2 
Ri + R 2 <n 2 + mi - rank(i7 2 i 

R 1 + R 2 <rank fj 11 . Hl2 



+ rank 



H< 



21 



2Ri + R 2 <ni + mi + rank 



21 u n,2 X m2 

rank(77 2 i) — rank(77i 2 ) 

7^21 #22 



ni Xmi 



#1 



i?i + 2i? 2 <^2 + m 2 + rank 



ni X mi ±± 12 

rank(i7 2 i) — rank(i7i 2 ) 

#11 H12 



0, 



21 u ri2Xm2 

rank(i7 2 i) — rank(i7 i2 ) 



(7) 
(8) 
(9) 
(10) 

#22 
#12 

(11) 



(12) 



(13) 



Before proceeding to the proof, we give an alternative 
presentation of the region specified in ([7])-(T3]). Firstly, de- 
note rank(i^j) by r^, i,j G {1,2} and let the singular 
value decomposition (SVD) of i7 i2 , R 2 \ be expressed as 
#i 2 = UiAiV^ and H 21 = U 2 A 2 V 2 T , where U U V U U 2 
and V 2 are n\ x m, m 2 x 777,2, n 2 x n 2 and mi x mi 
orthogonal matrices, respectively; Ai,A 2 are n\ x m 2 and 
n 2 x mi diagonal matrices with singular values of Hi 2 ,H 2 i 
on the main diagonal. Furthermore, Ai, U\ and V\ can be 
decomposed as follows: 



D 



12 



ri2X(m 2 -r 12 ) 



0, 



0(ni-ri 2 )xri2 u (ni -r 12 ) X (m 2 -ri 2 ), 



where 



Ai = 

D\2 € R ri2Xri2 is a diagonal matrix with the non-zero 
singular values of H\ 2 on its main diagonal. 
U x = [Uu U 10 ], where Uu € R niXri2 whose 
columns form an orthonormal basis for the subspace 
spanned by columns of i7 i2 , i.e. 1Z(Uu) = 1Z(Hi 2 ); 
U 10 e R n ^ x ( n ^- r ^) spans the null space of H? 2 , i.e 

n{u 1Q )=N{HT 2 ). 

V x = [Vn Vio], where V n e R™**^ whose 
columns form an orthonormal basis for the subspace 
spanned by rows of Hi 2 , i.e. H(Vu) = 1Z{H^ 2 )\ 



x(m 2 r 12 ) S p ans the null space of Hi 2 , i.e. 
K(V 10 )=M(H 12 ). 
Similarly, A 2 , U 2 and 
A 2 



as 

U 2 



[U21 u 20 ],v 2 



v 2 
D21 

®(n 2 -r 21 )xr 21 
[V21 V 20 \. 



can 



be decomposed 

0r 2 i x(m 1 -r 2 i) 



(n 2 -r 2 i)x(mi-r 2 i) 



■#11 
#21 



#12 

On 2 Xm 2 



Lemma 1: rank 
f2i + r i2 

Proof: Since U\,U 2 ,V\, V 2 are non- singular 



= rank(/7 1 T 77nl/2o) + 







and 



y 2 
Vi_ 

the matrix rank is unchanged with a left or right multiplication 



are non-singular as well. By using the fact that 



by a non- singular matrix, we have 
#12 



rank 
= rank 
= rank 



'n 2 X m 2 
T 



#11 

#21 

Vf 
ul 

UTHHV2 Ai 
A 2 

'ttT 
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(a) , 

= rank 





^#11^20 

D 2 i o 
= rank(C/2)#iiV2o) + r 21 + r 12 



D 12 





Lemma 2: rank 



rank((7 2 T #22^io) + 



where (a) follows by elementary row and elementary column 
operations with the fact that D\ 2 and D 2 \ are non- singular. ■ 
Following similar arguments as in Lemma [T] we have 
#21 #22 

_0niXmi #12 

r 2 \ + ri2 

With the above results, ([7])-(p3]) can be equivalently ex- 
pressed as 

Ri <rn (14) 
R2 <r 22 (15) 
Ri + R 2 <m + m 2 - 7*12 (16) 
Rx + R 2 <n 2 + mi - r 2 i (17) 
R x + i? 2 <rank((7 1 T #iiy 2 o) + rank((7 2 T # 2 2^io) 

+ r 2 i+ri2 (18) 
2R X + R 2 <m + mi + rank((7 2 T #22^io) (19) 
Rx + 2i? 2 <n 2 + m 2 + rank((7 1 T i7iiV 2 o) (20) 



A. Achiev ability 

This subsection gives the achievability proof of the capacity 
region given in Theorem [I] Given a rate pair (Ri,R 2 ) that 
satisfies the inequalities in (p4|)-([2Q|), we show that it is 
achievable by using linear precoding together with a specific 
type of Han-Kobayahsi rate- splitting. 

Firstly, with the SVD expressions for R\ 2 and #21 given 
previously, and by absorbing the orthogonal matrices into 
the input vector Xj and multiplying the output vector with 
Uj ', i,j E {l,2},i 7^ j, the channel model given in ([]]) can 
be equivalently written as 



y'i 
y 2 



#{ 1 x / 1 + Aix 2 

^ 2 x 2 + a 2X ; 



(21) 
(22) 



where yi = Ufyu y 2 = C/Jy 2 , xi = y/xi, x 2 = yi T x 2 , 
= /7i T #nV2 and # 22 = U 2 T H 2 2V 1 . The advantage of 
this equivalent channel model is that it results in diagonal 
interfering channel matrices, which is easier to deal with. 
Similar transformations have been used in [7 ]. To find the input 
signal vectors xi and x 2 , it is sufficient to determine x^ and 
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xj> since they are related by the nonsingular transformations 
given by xi = V^x^ and x 2 = Vix! 2 . 

Denote by di and d 2 the information-bearing symbols to 
be sent by s\ and s 2 , respectively, where di G R Rl and 
d 2 G IR^ 2 . Motivated by the rate- splitting technique used 
in the celebrated Han-Kobayashi schemes, we divide the R\ 
symbols in di into two parts: the common information d ic G 
R Rlc , which is decodable at both t\ and t 2 , and the private 
information d ip G which is decodable at t\ only. Then 

"di, 



we have di = 



and Ri — R\ c + R\ p . To map di to the 



mi -dimensional input vector x' 1? random spreading is applied. 
Specifically, let E\ c G 



!lXi?] 



E 



[m 1 -r 21 )xR lp 



be randomly and independently generated matrices, then the 
transmitted signal vector in the channel model pT) is given 
E d 

by x' = t; lc " lc . Note that x.[ is obtained by effectively 

precoding the information-bearing symbols di with a block- 
diagonal matrix with E\ c and E\ v on the block diagonals. The 
effect is that the common and private symbols d ic and d ip are 
constrained to the first r 2 i and last m\ — r 2 i components of 
x[, respectively. With such a rate- splitting and precoding, later 
we will show that the private symbols d ip will not affect the 
received signal vector y' 2 at t 2 , i.e., the inter-user interference 
caused by the private symbols is zero-forced; and the common 
symbols d ic can be decoded at both t\ and t 2 if the constraints 
given in (p4|)-([2Q|) are satisfied. 

Similar transmission scheme can be applied at s 2 , i.e., 
the information-bearing symbols d 2 G R R2 are split into 

lap € R R ^, and x' 2 ^ 2cd2c 



^2c 



L2c and d 



E 2p d 2p 



where 



E 2c G M ri2XjR2c and E 2p G R( m 2-ri 2 )xR 2p are randomly and 
independently generated matrices. 

The channel H' n in pT] ) can be expressed as 



T 



Hn[V2i V, 



20 



^11^11^21 ^11^11^20 
20 



Therefore, the output at t\ given in pT] ) can be written as 



yi = 



D 12 



Ei p di p 



P(n 1 -r 12 )xr 12 



ri 2 x(m 2 -r 12 ) 



(n 1 -r 12 )x(m 2 -r 12 )_ 

Ei c d lc 



E 2c d 2c 
E 2p d 2p 



UnHiiV 2 i UnHnVw D\ 2 
Ui HiiV 2 i Ui HnV 2 Q 



Ei p di p 
E 2c d 2c 



(23) 



( [23] ) clearly shows that the private symbol vector d 2p trans- 
mitted by s 2 does not affect y[ due to the block-diagonal 
precoding discussed previously. Although t\ is interested in 
recovering d ic and d ip only, ( [23] ) shows that decoding the 
common symbols d 2c is also necessary since the decoding 
process is equivalent to solving a system of linear equations 
with unknowns di c , d ip and d 2c . 



Similar arguments hold for t 2 as well, where we have 



y'2 



U 21 H 22 Vn U 21 H 22 Vio 
U 20 H 22 Vn U 20 H 22 Vi 



10 



^2cd2c 

E 2p d 2p 



D 



21 



{n 2 -r 21 )xr 21 



U 21 H 22 V±i U 21 
U 20 H 22 Vu U 20 H 22 Vio 



U 21 H 22 V±o 



^r 21 x(m 1 -r 21 ) 
(n 2 -r 21 )x(m 1 -r 2 

E 2c d 2c 



Eicdic 
Eipdip 



D 2 i 




E 2p d 2p 
^lcdic 



(24) 



To find a sufficient condition such that the system of linear 
equations given by ( [23] ) and ( [24] ) are uniquely solvable, the 
following results are shown to be useful: 

Lemma 3: (T2| Given the relationship y = Ax, where A G 
R pxl , x G R l and y G R p , then x can be uniquely determined 
from y if A is of full column rank, i.e., rank (A) = I. 

To simplify the presentation, let 



Mi 



UiiH\iV 2 i U^HnV^ Di 2 
U? HiiV 2 i U? HnV 2 o 



Eic 











Eip 











E 2c 



(25) 



Hence, ( [23] ) can be written as yi = Mi [dj c dj p dJ c ] T . 
According to Lemma[3] the receiver ti can successfully decode 
die, dip and d 2c if Mi is of full column rank. Next, we find a 
sufficient condition over the data rates Ri c , Ri p and R 2c such 
that Mi has full column rank. 

Lemma 4: Given Ai G R px/l , A 2 G R pxh and A 3 G 
R px/3 ,andlet^i G R hxkl ,E 2 G M /2X/C2 and£ 3 e R lsXks be 
randomly and independently generated, then the full column 
rank condition rankQAii^i A 2 E 2 A 3 E 3 }) = ki + k 2 + k 3 
holds with probability 1 if the following conditions are satis- 
fied: 

• ki < rank(Ai) 

• k 2 < rank(A 2 ) 

• k 3 < rank(A 3 ) 

• ki + k 2 < rankQAi A 2 }) 

• ki + k 3 < rankQAi A 3 ]) 

• k 2 + k 3 < rank([A 2 A 3 \) 

. ki + k 2 + k 3 <rank([Ai A 2 A 3 ]) 

Proof: Please refer to Appendix [B] ■ 
Consider Mi given in p5l, we have 



Mi 



^n^ii^i , , - 

^11^21 ' 



D 12 




E 



2c 



Recall that E lc 

xR 2c 



^r 21 xR 1% E ^ e K (mi-r 2 i)xi2i p and 

E 2c G R r ^ XK ^. By directly applying Lemma Q a sufficient 
condition for Mi to be of full column rank, and hence ( [23] ) is 
uniquely solvable, is given by 
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successfully decode d 2c , d 2p and d ic is given by 



Ric < rank 
= rank 
Ri p < rank 

R2c < 

Ric + Rip < rank 



Ui Q HuV 2 i 
r<mk(U?HnV 2 i) 

^11-^*11^20 
^10^11^20 

rank([/ 1 T i7 11 y 20 ) ( = } 
£12 

0(ni— ri 2 )xri2 

^11^11^21 ^11^11^20 
^10^11^21 ^10^11^20 

rank(t/ 1 T i7 11 y 2 ) ( = } r n 



mi - r 2i 
= r 12 



(26) 

(27) 
(28) 

(29) 



Ric + ^2c < rank 

(c) 



UiiHuV^i D12 
Ui HnV 2 i 0( ni _ ri2 ) xri2> 



Rip + i? 2c < rank 



(d) 



i?i c + + i? 2c < rank 



r i2 + rank(C/ 1 ^i7 n V 2 i) 

Ui^HnV^o D12 
Ui HnV 2 o 0( ni _ ri2 ) xri2 

n 2 + rank(f/ 1 7 i7iiV 20 ) 

U\\H UiiHhV 2 q 



(30) 



(e) 



ni 



(31) 

Di 2 


(32) 



where (a) follows from Lemma [5] in Appendix [A] as 

rank(/7 1 T i7 11 V 20 ) = rank(iJ n y 2 o) 
= rank(y 20 ) - dim(A/"(iJ n ) D ^(Vbo)) 
= rank(y 20 ) - dim(AT(i7 n ) ntf(H 21 )) 

= rank(y 20 ) = mi - r 2 i 

where (f) holds since C/i is full rank, and (g) follows the 
assumption given in ([5]). 

Moreover, (b) is true since XJ\ and V 2 are non- singular, 
(c) and (d) can be obtained by applying elementary column 
operations since Di 2 is nonsingular, (e) can be shown with 
elementary column operations together with similar proof as 
that for (a), i.e. 



rank 



^11^11^21 UiiHnV 2 o Di 2 
Ui HnV 2 i UiqHhV 2 o 



= rank ( [U^HnV 2 i U^HnV 20 ] ) + rank(£ 12 ) 
= Tank(U? H 11 V2) + r 12 
= rank^^n) + r X2 

= rank(iJn) - dim(AT((7 1 T ) D IZ(Hn)) + r 12 
= m - d3m(K(H 12 ) n ft(# n )) + T12 
= rn - (rn + r 12 - rank([i7n #12])) + r 12 
rank([i7n H 12 \) ^1 

By symmetry, a sufficient condition for receiver t 2 to 



i? 2c < rank([/ 2 T tf 22 Vn) (33) 

R2 P <m 2 - ri2 (34) 

Ric < T21 (35) 

R2c + < r 22 (36) 

R2c + Ai c < T2i + rank([/ 2 T ^ 22 y n ) (37) 

R2 P + i*i c < r 2 i + rank([/ 2 T tf 22 y 10 ) (38) 

R2c + R2 P + i?i c < ^2 (39) 

Since Ri = Ri c + Ri P and R 2 = R 2c + R2 P , the conditions 
on the data rate Ri and R 2 to ensure full decodability at the 
respective destinations can be obtained using Fourier-Motzkin 
Elimination over (|26}-([32} and ([33])-([39|). The detailed steps 
can be found in (13) and the resulted achievable rate region 
is given by fl4])-(|20]). This completes the achievability proof. 

To sum up, the precoding and decoding process in the 
proposed achievable scheme is depicted in Figj2j 
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(a) Precoding process at Si 
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(b) Decoding process at ti 

Fig. 2. Precoding and Decoding at s\ and t\ respectively 



B. Proof of the converse 

In this subsection, we give the converse proof of Theorem [T] 
Firstly, it is obvious that the rates for the two-user linear 
deterministic IC given by (p} are bounded by the single-user 
capacity, i.e. R 1 < ru,R 2 < r 22 . 

For notational convenience, let zi = i? 2 iXi, z 2 = i^i 2 x 2 , 
which are the interference terms observed at t 2 and ti, 
respectively. 

For ti to fully decode xi, it is obtained that the interfer- 
ence term z 2 = yi — if 1 1X1 can be uniquely determined 
as well. Since V10 spans the null space of Hi 2 , we can 
H12 



obtain rank 



w 



Therefore, if the term 
uniquely determined. As 



m 2 , which is of full column rank 

H12 



x 2 is given, then x 2 can be 
i7 i2 x 2 = z 2 is known, if a "genie" 



provides the value for V^x 2 to the receiver ti, then both xi 
and x 2 are decodable. Let H(-) denote the entropy, then the 
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sum rate must satisfy 



R 1 +R 2 <H 
H 



< rank 



yi 

^10 X 2 

#11 H12 

(m 2 -r 12 )xm 1 

#11 



H 



(m 2 -r 12 )xm 1 



12 

Vi T o 



< rank([i7n H 12 \) + rank ( [0 (m2 _ ri2)xmi 
= rii + m 2 - 7*12 

This completes the proof of (|9). ( fTU] ) can be proved similarly. 
To prove ( p"T| ), the following result is needed: 
Lemma 5: Let x be a random vector of dimension / x 

1, given A e R PlXl ,B e R P2Xl , then i7(Ax|Ex) < 

rank ^ ^ ^ — rank(5). 

Proof: Please refer to Appendix [C] ■ 
Since xi and z 2 can be simultaneously determined from yi 
at ti, we have iJ(xi,z 2 ) < H(yi). Furthermore, as xi and 
z 2 are independent, we have i7(xi,z 2 ) = #(xi) + #(z 2 ), 
which gives #(xi) < #(yi) — #(z 2 ). Similarly, it can be 
shown that if (x 2 ) < #(y 2 ) — H(zi). Therefore, we have 

R! + R 2 < tf(xi) + #(x 2 ) 
<#(yi)-#(z 2 ) + #(y 2 )-tf(zi) 
<H(y 1 ,z 1 ) - H{z 1 ) + (y 2 , z 2 ) - tf(z 2 ) 
=#(yi|zi) + #(y 2 |z 2 ) 

#11 #12 



(a) 

< rank 



+ rank 
Tank 



#21 

#21 #22 
#i 2 

#11 #12 
#21 



rank ([H 21 0]) 
-rank([0 # i2 ]) 



ri 2 - r 2i 



where (a) follows from Lemma [5] This completes the proof 
of £□}. 

To prove fT2] ), similar arguments can be used, i.e., 



2R 1 +R 2 < 2i7(xi) + i7(x 2 ) 

<2fT( yi ) - 2H(z 2 ) + H(y 2 ) - H( Zl ) 

<H( Yl ) + H( Yl , zi, z 2 ) - 2tf (z 2 ) + tf(y 2 ) - #( Zl ) 

=#(yi) + #(yi|zi, z 2 ) + #(y 2 |z 2 ) 

#11 #12 
#21 

. o # 12 

#21 #22 
#12 

#21 #22 
#12 



(b) 

< Hi + rank 



rank 



#21 







#12 



+ rank 
■rii + mi + rank 



ri2 



ri2 ~ r 21 



where (b) follows from Lemma [5] By symmetry, ( p"3] ) can be 
proved similarly. 

This completes the proof of the converse for Theorem [T] 



C. Discussion 

In this subsection, we show that the DoF region for the 
two-user Gaussian IC given in ^ is a special case of the 
capacity region for the linear deterministic IC specified in |7])- 
( p"3] ), which is obtained with Hij,i,j £ {1,2} being random 
(thus full rank) and independent. 

If all the channel matrices are full rank, we have rn = 
min{mi,ni}, r 22 = min{ra 2 ,n 2 }, r 12 = min{m 2 , rii}, 
and r 2 i = min{mi,n 2 }. Since they are also independent, 



rank 



#n 

#21 



#12 





rank 



#21 





#22 
#12 



min{mi + 



m 2 ,ni + n 2 }. Thus, ([7])-(T3]) reduce to 

Ri < min{mi , ri\ } (40) 
R 2 <min{ra 2 ,n 2 } (41) 
R\ + R2 <n± + m 2 — min{m 2 , ni} — max{m 2 , ni} (42) 
Ri + R 2 <n 2 + mi — min{mi, n 2 } — max{mi, n 2 } (43) 
Ri + R 2 <2min{mi + m 2 ,n 1 + n 2 } 

— min{m 2 , ni} — min{mi, n 2 } (44) 
2Ri + R 2 <mi + ni + min{mi + m 2 ,ri\ + n 2 } 

— min{ra 2 , n\ } — min{mi , n 2 } (45) 
Ri + 2R 2 <m 2 + n 2 + min{rai + m 2 ,n 1 + n 2 } 

— min{m 2 , n\ } — min{mi , n 2 } (46) 



If mi > n 2 , m 2 > ri\ and m\ + m 2 > ri\ + n 2 , (|40|-([46J 
reduce to 



i?i < min{mi, ni} 
R 2 < min{ra 2 ,n 2 } 
Ri + R2 < m 2 
Ri + R 2 < m\ 
Ri + R2 < ni + n 2 
2i?i + R 2 < mi + ni 
R x + 2i? 2 < m 2 + n 2 



(47) 
(48) 
(49) 
(50) 
(51) 
(52) 
(53) 



Note that ( |52| ) is implied by ( [47] ) and ( |50| ), and thus is 
redundant. Similarly, ( [53] ) is also redundant. The region formed 
by ([47])-([53]) is exactly the same as the that given in (|2| 
under the same conditions that mi > n 2 , m 2 > ri\ and 
mi + ra 2 > ni + n 2 . Other cases can be proved in a similar 
manner and thus are omitted here for brevity. 

IV. An Achievable Region for Double-Unicast 
Networks 

A double-unicast network can be represented by a directed 
acyclic graph G — (V, E) with two sources S — {si, s 2 } C V 
and two receivers T — {ti,t 2 } C V. Similar to the two-user 
ICs, si and s 2 are intended to send independent messages to 
t\ and f 2 , respectively, and inter-user interference is resulted 
since both pairs share the same network. 

Assuming that each edge is capable of carrying one symbol 
per time slot. By Merger's theorem, the minimum cut between 
sets *SjVi ^ S and T^ 2 C T is the number of edge 
disjoint paths from Sn ± to T^ 2 (denoted by kN 1 -N 2 X where 
Ni,N 2 C {1,2}. With random linear network coding per- 
formed at all intermediate nodes, the double-unicast network 
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TABLE I 

Dimension and Rank of Matrices 



Channel Matrix 


Size 


Rank 


H U 


&12-1 X fcl-12 


k\-i 


#12 


&12-1 X /C 2 -12 


k2-i 


H21 


&12-2 X fci_i2 


ki-2 


H22 


&12-2 X /C 2 -12 


k2-2 








&12-1 X (fci-12 + fc 2 -12) 


k\2-i 




H21 H22 




&12-2 X (fci-12 + ^2-12) 


k\2-2 




Hn 
H21 




(&12-1 + ^12-2) X fci_i2 


ki-12 




H\2 

H22 




(&12-1 + ^12-2) X fe 2 -12 


&2-12 



T?pcrion 1 


T?pcrion 


Ri < k 12 -i - k 2 -i 


Ri < h-! 


R2 < k 12 -2 ~ h- 2 


R 2 < min{A;i2-i, &12-2} - k X - 2 
Ri + R2< rank( [H u H 12 M 2 ] ) 


Region 3 




i?i < min{A:i2-i, A:i2- 2 } - k 2 -i 


R 2 < k 2 -2 




Ri+R2< rank( [H 21 M 1 H 22 ] ) 



can be modeled as a two-user linear deterministic IC where 
the transition matrices are determined by the network topology 
and the coding coefficients chosen at each intermediate node. 
Without loss of generality, the dimension and the rank of 
the transition matrices are represented by the min-cuts of the 
network, i.e., kN 1 -N 2 as shown in Table [I] [ 11 ]. 

With Theorem [T] the rate pair (Ri,R 2 ) is achievable if the 
following conditions are satisfied: 



Ri <h-i 
R 2 <k 2 _ 2 
Ri + R2 <ki 2 -i + &2-12 ~ k 2 -i 
Ri + R2 <ki2-2 + fci-12 - ki-2 
Hu H\ 2 
H21 
- k 2 -! 



(54) 
(55) 
(56) 
(57) 



where Mi and M 2 are some mapping matrices of size 
ki-i 2 x Ri and k 2 -\ 2 x R 2 , respectively. It is shown in JTT] 
that for the low interference case, the rate pairs that are in the 
convex hull of Regions 1, 2 and 3 are achievable, and for the 
high interference case, the convex hull of Regions 2 and 3 is 
achievable. 

Lemma 6: The proposed achievable region specified in 
([54])-([6Q]) for the double-unicast networks is larger than the 
convex hull of Region 1, 2 and 3. 

Proof: The region specified in ([54]) -([60]) is convex. There- 
fore, to show that it is larger than the convex hull of Regions 1, 
2 and 3, it is sufficient to show that all the individual Regions 
1, 2 and 3 are within the proposed region. 

To prove that Region 1 is within the proposed region, we 
show that any rate pair (Ri,R 2 ) in Region 1 satisfy ([54]) -([60]). 



Rl < &12-1 



(a) 
1 < 



(61) 



+ rank 



H21 




H22 
H12 



(58) 



Ri + R2 <rank 

- &1-2 

2Ri + R 2 <k 12 -i + &i_i2 + rank 

- &4_ 2 - &2-1 

Ri + 2R 2 <k 12 -2 + &2-12 + rank 

- /ci_ 2 - k 2 -i 



Note that although ([7])-([T3]) give the capacity region of the 
linear deterministic IC, (|54])-(|60]) only give an achievable re- 
gion for double-unicast networks since random linear network 
coding at all the intermediate nodes may be sub-optimal. 
Nevertheless, as random network coding can be practically 
implemented due to its simplicity, it is widely used in practice 



A. Comparison with Existing Results 

In this subsection, the achievable region given by ([54])-([6Q]) 
is compared with that in [11] and [8], which both consider the 
double-unicast networks. 

1) Comparison with [11]: In (TT), two scenarios are 
considered separately, i.e., the low interference case with 
ki-2 + k 2 -i < min(&i2-i, &12-2) and the high interference 
case with /ci_ 2 + k 2 -i > min(&i2-i, ^12-2)- They give rise 
to the following achievable regions: 



where (a) follows ki 2 . 
rank(i7n) + rank(i7i2) - 
R2 < k\ 2 - 2 - ki- 2 < 
inequalities are satisfied: 



_! = rank([F n H 12 ]) < 
= ki-i + k 2 -i. Similarly, we have 
k 2 - 2 . Furthermore, the following 



H21 


H22 




Ri + R2 < k 12 -i - k 2 -i + k 12 


_ 2 - &1-2 (62) 





H12 


) 


< ki-i + fci2-2 - fa- 


2 (63) 






(59) 


< fei_i2 + k 12 -2 - k x 


-2 (64) 


H21 


H\2 




) 

(60) 


Thus, ([57]) is satisfied. Similarly, Ri + R 2 
k 2 -i can be proved. With the inequality g 
following relations, 


< ^12-1 + k 2 - 12 - 
iven in ([62]) and the 



rank 
rank 



#11 


H\2 


H21 





H21 


#22 





H\2 



> rank ([tfii H 12 ])=k 12 -i (65) 

> rank ( [H 21 H 22 ] ) = k 12 - 2 (66) 



58]) can be proved. Furthermore, ([59]) is satisfied since 

(P) 

2Rx + R 2 <ki-\ + k\2-i - k 2 -i + ^12-2 - ^1-2 

<kl-l 2 + &12-1 - k 2 -i + fei2-2 - h-2 
H21 H22 

H 12 

- k X -2 



<rank 



+ fei_i2 + Ali2-l 



'2-1 



where (b) follows from ( |6T] ) and ( [62] ), and (c) is true due to 
( [66] ). By symmetry, it can be shown that ( [60] ) is also satisfied. 
This completes the proof that Region 1 is within our proposed 
region. 
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In order to show that Region 2 is also within our proposed 
region, the following region (denoted as Region 2') is defined: 



Ri < h-i 

R 2 < min{/ci2-i, k 12 -2} 
Ri+R 2 < k 12 -i 



'1-2 



(67) 
(68) 
(69) 



It is obvious that Region 2' is no smaller than Region 2 since 
rank([#n H 12 M 2 }) < rank ([#n H 12 \) = k 12 -i, VM 2 . 
Then it is sufficient to prove that any rate pair (Ri,R 2 ) in 
Region 2' satisfy ([54])-([6Q]>, which is given as follows: 



Ri <h-i 

R 2 < min{A:i2-i, &12-2} - k X - 2 

(a) 

<k\ 2 - 2 — k\- 2 < k 2 - 2 
Ri + R2 <h-i + min{/ci2_i, ^12-2} - k\_ 2 



(b) 

</Ci_i2 + &12-2 

(c) 



1-2 



Ri + R2 <h 2 -i < k 12 -! + (k 2 -i2 ~ k 2 -i) 



(d) 

Ri + R2 <&i2-i < rank 



(e) 

<rank 



#11 #12 

#21 0fcl2-2Xfc 2 _i 2 

#21 





+ rank 



#11 #12 

#21 

- &2-1 - &1-2 

2Ri + R2 <ki 2 -i + /ci-i < A;i2-i + &1-12 
(f) 

< &12-1 + &1-12 + rank 



#22 
#12 





H21 


#22 


( 





#12 



- k 2 -i - k X - 2 

Ri + 2R 2 <k± 2 -i + min{A;i2-i, &12-2} _ ^1-2 

<k± 2 -i + &12-2 - ki-2 

#12 




(9) 

< rank 



(h) 

< rank 



2-12 



#11 
#21 

#11 #12 
#21 

&2-1 



+ k 12 - 2 
+ &12-2 



&1-2 
&1-2 



where (a) follows from similar arguments as ( |6T] ); (b) is true 
since k\-\ < /ci-12; (c) is satisfied due to k 2 -\ < &2-12; 
(d) is obtained by using ( [65] ), and (e) follows from Lemma [2] 
which gives 



rank 



7^21 #22 
#12 



&2-1 - &1-2 = rank([/^o#22Vio) > 



(f) can be shown with Lemma [2] in a similar manner; (g) is 
true due to ( [65] ); and (h) is satisfied since fe-12 > k 2 -\. This 
completes the proof that Region 2' is within the proposed 
achievable region. Therefore, Region 2 must be within our 
proposed region too. By symmetry, Region 3 can be shown to 
be within the proposed region as well. The details are omitted 
for brevity. 

This completes the proof of Lemma [6] ■ 



2) Comparison with [8]: The achievable region derived in 
(8) is given by the convex hull of the following two regions: 



Region 4 

Ri < 

2Ri + R2< k 2 -2 



Region 5 

R 1 + 2R 2 < fci_i 
R 2 < k 2 -2 



Lemma 7: The proposed achievable region specified in 
([54]) -([60]) for the double-unicast networks is larger than the 
convex hull of Region 4 and Region 5. 

Proof: Since the proposed region is convex, it is sufficient 
to show that both Region 4 and Region 5 are within the 
proposed region. For any rate pair (Ri,R 2 ) in Region 4, ( [54] )- 
([601 are satisfied since 



Ri <ki-i 

R2 <k 2 -2 ~ 2Ri < k 2 -2 
Ri + R2 <k 2 -2 - R\ < k 2 -2 < k 12 -2 

<k\2-2 + &1-12 - &1-2 

R\ + R2 <k 2 -2 - R\ < k 2 -2 < k 2 -\2 
<k 2 -i2 + k 12 -i - k 2 -i 

#21 



Ri + R2 <k±2-2 < rank 



(a) 

< rank 



#21 




#22 
#12 



+ rank 



- k 2 -i 

2i?i + R 2 <k 2 -2 < k 12 -2 < rank 



#21 




#22 

#12 

#11 
#21 

#22 
#12 



#12 





<rank 



1-12 



fei. 





#11 


#12 


( 


#21 






#21 #22 

_ # 12 
+ (&12-1 - k 2 -i) 
R x + 2R 2 <2k 2 - 2 < k 2 -i2 + A;i2- 2 
(P) 

<k 2 -i 2 + k 12 -2 + rank 
- /ci_ 2 - k 2 -i 

where (a) and (b) follows from Lemma [T] 

By symmetry, any rate pairs (Ri,R 2 ) in Region 5 can be 
shown to satisfy ([54]) -([60]). This completes the proof of Lemma 

m ■ 

3) Discussion: It was pointed out in fTT) that for certain 
network topologies, there exist some rate pairs that are achiev- 
able by the scheme in (TTJ but not achievable by the scheme 
in (8), and vice versa. According to Lemma [6] and Lemma 
[71 our region is larger than both regions given in fTT| and 
|8). Therefore, it can be concluded that our proposed region 
is strictly larger than both of them. Actually, there exists some 
network instance (for example, the network shown in Fig. [3]) 
where our proposed region is strictly better than the union of 
the region given in (TT) and [8]. 

B. Implementing the Network Code in Finite Field 

To achieve the region specified in ([54]) -([60]), the standard 
MIMO SVD technique has been used. SVD is well defined for 
real matrices but not for matrices in finite filed. Therefore, to 
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achieve the proposed region for the double-unicast networks 
within certain finite filed, some slight modifications for the 
previously discussed achievability scheme is required. To this 
end, we need to find matrices that have similar properties in 
the chosen finite field as the orthogonal matrices obtained via 
SVD. Specifically, assuming that all the transition matrices are 
chosen from a finite field F q , where q is power of prime, the 
following matrices are defined: 

. Let Un e Fq 12 - 1 ** 2 - 1 be a basis for K(H 12 ). 



Let U 10 e Fc 



fei2-l X(fei2-l-fe 2 -l) 



1} be a basis for JV(H{ 2 ). 



Let V u e f} 2 - 12 ^ 2 - 1 be a basis for 1Z(Hf 2 ). 



. LetyxoGF^ 2 - 12 ^^- 12 ^ 2 - 1 



be a basis for J\f(Hi 2 ). 



Then, we have 



uTo 



H 12 [V n V 10 ] 
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L0(fc 12 _ 



k 2 -i)xk 2 -i 
ko- 



0i 



x(fc : 



(/ci2-l-fc 2 -l)x(/C2-12-fc2-l). 



x k< 



where U^H^Vu G i^- 1 ^ 2 " 1 is of full rank, although it 
may no longer be diagonal. For notational convenience, denote 
by D 12 . 

Similarly, we can obtain C/21, U20, V21, V20 and D 2 i — 
Uji ^21^21 in finite field. When the field size is sufficiently 
large, Lemma [4] holds with high probability (refer to Corollary 
TV Therefore, following similar arguments as that in section 
III] the region specified in ([54])-([6Q]) can be achieved in finite 
field as well. 

Example 1: Consider the network shown in Fig[3la) where 
each edge has unit capacity. Assume the field siztj^is given 
by q — 7. With random network coding performed at interme- 
diate nodes, one possible realization of the effective channel 
matrices for the equivalent two-user linear deterministic IC are 
given by 



Hy 



21 





"2 


0" 


; H12 — 


"2 


1 0" 




2 


3 


2 


1 1 




"1 


0" 




"1 







2 


3 


; #22 — 


2 


1 




2 


3 




2 


1 1 



Then the achievable region (after removing redundant in- 
equalities) specified in ([54])-([6Q]) for this example is given by 

R x < 2 
Ri + R2 < 3 
2Ri + R 2 < 4 

which is plotted in Fig|3Jb). Note that the gray area denotes 
the achievable region given by (TT) and (5) with time- sharing. 
Next, we give the specific precoding scheme to achieve the 
rate pair (Ri,R 2 ) — (1,2) by following the achievability 
scheme given in Section 



III 



_ Let di dn, d 2 [d 2 i ^22] ■ 
Then, following (|26])-(|31|) and ([33]) -([39]), the private- common 
rate splitting is given by d ic = du,di p — 0,d 2c = d 2 i and 
where denotes an empty vector. 



<*2p 



^22 ■ 



2 A commonly used field size for random network coding is 2 8 . Here, a 
small field size is chosen for illustration purposes. 




R- 




(a) (b) 

Fig. 3. (a) An example of double-unicast network (b) Achievable region 

Assume that the following matrices are randomly generated, 
which will be applied to map the data symbols to the trans- 
mitted vectors: 



Ey 







f2l 


CO 1 


; E 2c — 


CO 1 



; E 2p 



3] 



E/10, Vii, Vio can be determined as well: 



Uy 



1 
1 1 





"2 


2" 




Y 


; U10 = 0; Vu = 


1 


1 


;Fio = 


5 







1 








Similarly, we can find U21, U20, V21, V, 



20 



C/21 = 



1 

2 3 
2 3 



C/20 = 



V21 = 



1 2 
3 



F 22 = 



Thus, the data transmitted at s 1 is given by 



xi = V 2 iE lc d lc 
Similarly, 

x 2 = [Vii V 10 ] 



1 2 
3 



dy 



3dn 
2du 



E2 c d.2c 
E2pd.2 p 



"2 


2 


1" 




"2^21" 


1 


1 


5 




3d 2 i 





1 







_3<i 2 2_ 



3d 2 i + 3d 2 2 
5d 2 i + d 2 2 
3^21 



yi #n 

y 2 — i^22 



3dn 
2dn 



6dn + M 2 i 



Therefore, the data received at t\ and t 2 are given by 

3d 2 i + 3d 2 2 

+ i7 i2 5d 2 l + <^22 

3d 2 i 

3d 2 i + 3d 2 2 
5d 2 l + d 2 2 + H 2 i 

3d 2 i 

3d 2 i + 3d 2 2 + 3dn 
5 (in 

With matrix inverse, both t\ and t 2 can recover their desired 
symbols. 



3d u 
2d u 
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V. Conclusion 

In this paper, the capacity region of the two-user linear 
deterministic IC is derived, where the result is given in terms 
of the rank of the transition matrices. Our result is applicable to 
the scenarios where the channel matrices are correlated and/or 
rank deficient. To achieve the rate pairs in the capacity region, 
we combine the standard MIMO SVD technique and the idea 
of common-private rate splitting, based on which a simple 
linear precoder is developed. Moreover, we show that this 
linear deterministic IC can be used to model the double-unicast 
networks when random network coding is performed at all the 
intermediate nodes. Therefore, the capacity results derived are 
utilized to obtain an achievable region for the double-unicast 
networks, and it is proved that the region is strictly larger than 
the existing results in the literature. 

However, there still exists a gap between our achievable 
region and the capacity of the double-unicast network as 
random linear network coding may be sub-optimal. One pos- 
sible future work is to find a better network coding strategy 
by optimizing the transition matrices subject to the topology 
constraint, instead of using simple random network coding, 
such that the proposed achievable region is maximized. 

Appendix A 
A Useful Lemma 

Lemma 8: |l2) Let A e M px/ and 5 e 
rank(AB) = rank(A) -dim(jV(A T )n1Z(B T )) = rank(5) 
6hn(jV(A)nn(B)) 

Proof: Refer to pp.126 in (12). 



oo. 



Ixk 



Appendix B 
Proof of Lemma[4] 

As the real field can be considered as F q with q 



TC7>xZ 2 
q 

q 



and 
and 



Lemma [4] is true if the following corollary holds. 

Corollary 1: Given A x e F£ xZl , A 2 e 
A 3 e W q xl \ and let E 1 e W% xk \ E 2 e 
E 3 <G ¥ l q 3Xks be uniformly and independently generated, then 
rank([Ai^i A 2 E 2 A 3 E 3 }) = k\ + k 2 + k 3 holds with 
probability approaching to 1 when q —> 00, if the following 
conditions are satisfied: 

• k± < rank(Ai) 

• k 2 < rank(A 2 ) 

• k 3 < rank(A 3 ) 

. k x + k 2 < rank([Ai A 2 ]) 
. £4 + k 3 < rankQAi A3]) 

• k 2 + k 3 <rank([A 2 A 3 ]) 

. k x + k 2 + k 3 <rank([Ai A 2 A 3 ]) 

Proof: Before proving Corollary [T] we need to establish 
some useful facts. 

Fact 1: fl5[| Let Vi(q) denote the vector space of di- 



mension I over the finite field F, 



the number of distinct k- 

I 
k 



dimensional subspaces of Vi(q), a quantity denoted by 

-i) 



q 



is 



k-l 



n 



Fact 2: Given a matrix A £ F^ x/ , let E be a random matrix 
uniformly generated from ¥ lxh , where k < rank(A), then 
Tsaik(AE) — k holds with probability approaching to 1 when 

q — >• 00. 

Proof: According to lemma [Ij rank(AE) = rank(^) — 
dim(1Z(E) fl J\f(A)). Since the elements in E are uni- 
formly and independently generated from the finite field ¥ q , 
Pr{rank( J E) = k) — nJ=o ~ g 1 ^ 1 ) anc * ^ approaches to 
1 as q — > 00. Therefore, to prove Fact [2] we only need to 
show that lim^oo Pr{dim(ft(£) n AT (A)) = 0} = 1. 

Provided with rank(I^) = k, 11(E) can be viewed as a 
subspace of Vi (q) chosen uniformly from all the subspaces of 
dimension k. Note that the null space of A, Af(A), is another 
subspace of Vi(q) with dimension I — rank(A). Thus, the 
probability that 1Z(E) has non-empty intersection with Af(A) 
is given by 



Pi{dim(1Z(E) HJV(A)) > 1} = 



'I - rank(A) 








1 




■ q 


k-l 


q 




T 










k_ 


q 







(q k - l)(g(-rank(A) _ ^ 



As a result, lim^ 

limg^QQ ran k(A)-fc+i 



Pi{dim(1Z(E) n AT (A)) > 1} = 
= since k < rank (A). Therefore, 
lim 9 _>oo Pr{dim(1Z(E) HJV(A)) = 0} = 1. This completes 
the proof of Fact [2] ■ 
Fact 3: Given matrices A 1 G ¥^ xl \ A 2 e F^ x/2 , let 
Ei and E 2 be two matrices uniformly and independently 
generated from ¥ l q lXhl and ¥ l q 2Xh2 , respectively. When q —> 
00, T8iBk([AiEi A 2 E 2 ]) — ki + k 2 holds with probability 
approaching to 1 if the following conditions are satisfied: 

• ki < rank(Ai) 

• k 2 < rank(A 2 ) 

• h + k 2 < rank([Ai A 2 }) 

Proof: For notational convenience, denote rank(Ai), 
rank(A 2 ) and dim(1Z(Ai) D 1Z(A 2 )) by 7*1, r 2 and r i2 
respectively. 

rank([A 1 ^ 1 A 2 E 2 ]) 

= rank(Ai^i) + rank(A 2 £ 2 ) - dim(ft(AiJ57i) fl 1Z(A 2 E 2 )) 

( = } h + k 2 - dim(7e(AiJS7i) n 1Z(A 2 E 2 )) 

where (a) follows from Fact [2] as k\ < n and k 2 < r 2 . 
Therefore, to prove Fact [3] we only need to show that, 
when q -> 00, dim(ft(AiJ57i) fl U(A 2 E 2 )) = holds with 
overwhelm probability. Note that IZ(AiEi) can be viewed as 
a random subspace of H(Ai) with dimension k\. Similarly, 
1Z(A 2 E 2 ) can be viewed as a random subspace of 1Z(A 2 ) 
with dimension k 2 . Therefore, the probability that 1Z(A\Ei) 
has non-zero intersection with 1Z(A 2 E 2 ) is given by 

Pr{dim(^(A 1 ^ 1 ) D1Z(A 2 E 2 )) > 1} 
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As a result, lim^oo 
lim 



-l)(gr a _l)( g _l) 

¥Y{&\m(K{A 1 E 1 )C\K{A 2 E 2 )) > 1} = 
g^oo ?) , + ,,.,+-, — 0, where the last equality follows since 
&i + fa < rank([Ai A 2 ]) = rank(Ai) + rank(A 2 ) - 
dim(1Z(Ai) n 7£(A 2 )) = ri + r 2 — r i2 . Therefore, we can 
conclude that, with overwhelm probability, dim(1Z(AiEi) n 
ft(A 2 £ 2 )) = 0. This completes the proof of Fact [5] ■ 
Now, we are ready to prove Corollary [T] For brevity, denote 
rank(Ai) by r», i G {1, 2, 3}, and dim(ft(^)nft(A,-)) by nj, 
i,j e {1, 2, 3}, i 7^ j. Therefore, rank[A^ A,-] = r^+Tj — . 

Assume that the given conditions in Corollary [T] are sat- 
isfied, according to Fact [5] and Fact [5] dim(7£(A^)) = 
ki,i G {1,2,3} and dim(ft(A^) n ^(A^)) = °> € 
{l,2,3},i 7^ j holds with probability approaching to 1. 
Therefore, we have 

rank([A 1 ^ 1 A 2 E 2 A 3 E 3 ]) 
=rank ( [A^ A 2 E 2 ] ) + rank (A 3 E 3 ) 

- dim {{H{A 1 E 1 ) + ft(A 2 £ 2 )) n K{A 3 E 3 )) 
=rank (A^i) + rank (A 2 E 2 ) + rank (A 3 £ 3 ) 

-dim {n{A 1 E 1 )nn{A 2 E 2 )) 

- dim ((7e(Ai£7i) + K{A 2 E 2 )) n ft(A 3 £ 3 )) 

=fci + fc 2 + fc 3 - dim ((ft(Ai#i) + 1Z(A 2 E 2 )) n ft(A 3 £ 3 )) 

Therefore, to prove Corollary [T] it is sufficient to show 
that dim ((IZiA^) + 1Z(A 2 E 2 )) n1Z(A 3 E 3 )) = holds 
with overwhelm probability. Denote the dimension of the 
intersection between (1Z(A 1 E 1 ) + K(A 2 E 2 )) and 1Z(A 3 ) by 
a, i.e., a = dim((ft(Ai#i) + ft(A 2 £ 2 )) n ft(A 3 j). The, 
the probability that 7£(A 3j E 3 ) has non-empty intersection with 
+ 1Z{A 2 E 2 )) is given by 

Pr{dim ((^(Ai^i) + ft(A 2 £ 2 )) H ft(A 3 £ 3 )) > 1} 
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lim^oo Pr{dim ((ft(Ai#i) + ft(A 2 £ 2 )) n ft(A 3 £ 3 )) > 
1} = if a < r 3 — k 3 . Therefore, in order 
to prove Corollary [T] it is sufficient to show that 
a = dim((ft(AiJ57i) + 1Z{A 2 E 2 )) n U(A 3 )) < r 3 - k 3 
holds with overwhelm probability. 

a =4xm((K(A 1 E 1 ) + 1Z{A 2 E 2 )) n ft(A 3 )) 
=rank ( [AiJSi A 2 £ 2 ] ) + rank(A 3 ) 

-rank([AiJ57i A 2 £ 2 A 3 ]) 
=ki + k 2 + r 3 - rank ( [A 1 E 1 A 2 E 2 A 3 ] ) 
=fci + k 2 + r 3 - rank ( [Ai^i A 3 ] ) - rank(A 2 £ 2 ) 

+ dim(^(A 2 ^ 2 ) n {n{A 1 E 1 ) + n(A 3 ))) 

=fa + r 3 - rank ( [Ai#i A 3 ] ) 

+ dim(ft(A 2 £ 2 ) n (UiAtEt) + ft(A 3 ))) 

=dim(7^(AiJ57i)n^(A 3 )) 

+ dim(n(A 2 E 2 ) H (^(Ai^i) + ft(A 3 ))) (70) 



Next, we proceed to calculate dim(1Z(AiEi) D 7£(A 3 )) and 
dim(ft(A 2 £ 2 ) H (ft(Ai#i) + ft(A 3 ))). For notational conve- 
nience, denote 7 = rank([Ai A 2 A 3 ]) and define (g) + = 
max{g, 0}. 

Fflc^: Um q ^ 00 PT{^m(n(A 3 )r\H(A 1 E 1 )) = (k 1 -r 1 + 
r i3 ) + } = 1 

Proof: The probability that the dimension of the intersec- 
tion between 1Z(A 3 ) and IZ(AiEi) is greater or equal to j is 
given by 



Pr{dim(^(A 3 )n^(A 1 ^ 1 ))> j)} 
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=0 (g' 



1) 



Therefore, lim 9 _^oo Pr{dim(^(A 3 ) n 7^(AiJ57i) > j)} = 



j +(fei+ri 3 -Ti)j 



• If j > (A* 



ri +ri 3 ) 1+ , lim^oo Pr{dim(^(A 3 ) D 11^^)) > j)} = 0. 
For any E\ e F l q lXkl , it can be verified that dim(7^(A 3 ) n 
H(AiE\)) > (ki — r\ + ri 3 ) + . Thus, we can conclude that 
lim^ 00 Pr{dim(^(A 3 )n^(A 1 ^ 1 )) = {fa-r^r^} = 1. 

■ 

Following similar arguments as that in the proof of Fact [4] 
with overwhelm probability, we have 



(71) 



dim(ft(A 2 £ 2 ) n (7fc(Ai#i) + n(A 3 ))) 
={fa - r 2 + dim(^(A 2 ) n + ^(A 3 )))) + 

Furthermore, 

dim(^(A 2 ) n (^(Ax^x) + 7e(A 3 ))) 
=rank(A 2 ) + rank ( [Ax^x ^3] ) - rank ( [AxEx A 2 A 3 ] ) 
=r 2 + rank(Ax^x) + rank(A 3 ) - dim(^(A 3 ) n 1Z(A 1 E 1 )) 

-rank([Ax^x A 2 A 3 ]) 
=r 2 + k x + r 3 - (Aix - n + r i3 ) + - rank ( [AiJ57i A 2 A 3 ] ) 
=r 2 + k± + r 3 - (A;i - n + r i3 ) + - rank(Ai^i) 

- rank ([A 2 As]) + dim(^(A 1 ^ 1 ) n (^(A 2 ) + ^(A 3 ))) 
=r 23 - (^ - n + r 13 ) + + dim(^(Ai^i) fl (^(A 2 ) + ^(A 3 ))) 



(ft) 



r 23 - (h - n + r i3 ) + 

+ (*i - n + dim(^(Ai) n (7^(A 2 ) + ^(A 3 )))) + 
r 23 - (A;i - n + r i3 ) + + (A;i + r 2 + r 3 - r 23 - 7)" 



(72) 



where (b) follows from the similar arguments as that in the 
proof of Fact [4] By substituting ([72) into ([71]), we get 

dim(ft(A 2 £ 2 ) H (ft(Ai#i) + ^(A 3 ))) = (fe - r 2 + r 23 
- - n + r i3 ) + + (A:i + r 2 + r 3 - r 23 - 7) + ) + 



(73) 



With Fact [4] and by substituting ( [73] ) into ( [70) , we have 

a =dim((K(A 1 E 1 ) + ^(A 2 ^ 2 )) n K(A 3 )) 

=(h - n + r i3 ) + + (A; 2 - r 2 + r 23 - (A;i - n + r i3 )" 
+ (A:i+r 2 + r 3 -r 23 - 7 ) + ) + 
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The remaining task is to show that a < r 3 — fa. Following 
cases are considered. 
Case I: fa — r\ + r i3 < 

In this case, a = (k 2 -r 2 + r 23 + (fa + r 2 + r 3 -r 23 -7) + ) + - 
If a = 0, the result holds trivially as fa < r 3 — rank (A3). 
Therefore, we only need to show that fa — r 2 + r 2 3 + (fa + 
r 2 +r 3 -r 23 -7)+ < (r 3 -fa). If fa+r 2 + r 3 -r 23 -~f < 0, it 
reduces to k 2 — r 2 + r 23 < r 3 — /c 3 , which is equivalent to the 
given condition k 2 + fa < r 2 + r 3 — r 23 = rank([A 2 A 3 ]). 
On the other hand, if fa + r 2 + r 3 — r 23 — 7 > 0, it reduces to 
&2 + (&i —7) < — fa, which is equivalent to the given condition 
fa + fa + /C3 < 7 = rankQAi A 2 A3]). Therefore, we 
conclude that a < (r 3 — fa) is true when £4 — 7*1 + r i3 < 0. 
Case II: fci — ri + r i3 > 

In this case, to prove a < (r 3 — fa), it is sufficient to show 
that (fa - r 2 + r 23 - &i + n - r i3 + (fa + r 2 + r 3 - r 23 - 
7 )+)+ < ( ri + r 3 - n 3 ) - (fa + /c 3 ). As A:i + fa < n + r 3 - 
ri3 = rank([Ai A 3 \), it holds trivially if the left hand side 
is reduced to zero. Therefore, we only need to show (fa +r 2 + 
r 3 - r 23 - 7)+ < (r 2 + r 3 - r 23 ) - (/c 2 + fc 3 ). Again, it is true 
when the left hand side is zero as k 2 -\-fa < rank([A 2 A3]) = 
r 2 + r 3 — r 23 . On the other hand, when the left hand side is 
greater than zero, it is further reduced to fa + fa + fa < 
7, which is exactly the same as the last condition given in 
Corollary [T] 

This completes the proof of Corollary [T] and hence Lemma 

m ■ 

Appendix C 
Proof of Lemma[5] 

This lemma is proved by similar arguments as that in section 

( \ A 

III of [16]. Denote rank(A), rank(£?) and rank 



by 

TA,rB and tab respectively. Let N\ be a matrix of sfze / x 

( \A 

(I — ^ab) whose column vectors form a basis for N 



Then we can find a matrix A^ 2 of size / x (tab — tb) such 
that the column vectors of N\ and N 2 form a basis for Af(B). 
Moreover, let N 3 eR lxrB , be the basis of 11(B). Therefore, 
[Ni N 2 N 3 ] spans the input space and we can find x' such 
that x = [iVi N 2 N 3 ] x r . Thus, 

i7(Ax|5x) (74) 
= H (A[N X N 2 N 3 ]x' I B[N X N 2 N 3 ]x f ) 
= H([0 lx(l _ rAB) AN 2 AN 3 ]x f \[0 BN 3 ]x f ) (75) 



Write x r 



x^ 



. ( [75] ) can be written as: 



H(Ak \Bx) = H (AN 2 x f 2 + AA^ 3 x 3 | BN 3 x f 3 ) 

As 11(B) = 1Z(N 3 ), dim(jV(B) n 1Z(N 3 )) = 0. According 
to Lemma [Ij rank(5A^ 3 ) = A^ 3 = tb- Thus, X3 is uniquely 
determined by BN 3 x 3 and we have 

H(Ayi \Bx) = H (AN 2 x' 2 + AN 3 x' 3 \ BN 3 x' 3 ,x 3 ) 
= H(AN 2 x f 2 \BN 3 x f 3 y 3 ) 

< H(AN 2 x. f 2 ) < rank(AAT 2 ) 

< rank(A^ 2 ) r A B ~ r B 



Thus, the lemma follows. 
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